Apple In My Eyes (AIME): Liveness Detection for Mobile Security Using Corneal Specular Reflections

The demand for contactless biometric authentication has significantly increased during the COVID-19 pandemic and beyond to prevent the spread of Coronavirus. The global pandemic unexpectedly affords a greater opportunity for contactless authentication, but iris and facial recognition biometrics have many usability, security, and privacy challenges, including mask-wearing and Presentation Attacks (PA). Mainly, liveness detection against spoofing is notably a challenging task as various biometric authentication methods cannot efficiently assess the real user’s physical presence in unsupervised environments. Although several face anti-spoofing methods have been proposed using add-on sensors, dynamic facial texture features, and 3D mapping, most of them require expensive sensors and substantial computational resources, or fail to detect sophisticated 3D face spoofing. This paper presents a software-based facial liveness detection method named “Apple in My Eyes (AIME).”AIME is intended to detect the liveness against spoofing for mobile device security using challenge-response testing. AIME generates various screen patterns as authentication challenges, then passively detects corneal-specular reflection responses from human eyes using a frontal camera and analyzes the detected reflections using lightweightMachine Learning techniques. AIME system components include Challenge and Pattern Detection, Feature Extraction and Classification, and Data Augmentation and Training. We have implemented AIME as a cross-platform application compatible with Android, iOS, and the web. Our comprehensive experimental results reveal that AIME detects liveness with high accuracy at around 200 ms against different types of sophisticated PAs. AIME can also efficiently detect liveness in multiple contactless biometric authentications without any costly extra sensors nor involving users’active responses. IEEE

Crowd Safety Sensing (CroSS) for the Post Pandemic Era

Due to its long incubation period, aggressive asymptomatic transmission, and new mutations of the virus, COVID-19 is causing multiple pandemic waves worldwide. Despite recent vaccination, social distancing, and social restriction efforts, false negatives, and dormant positives can make pandemics challenging to restrain. In addition to rapid vaccination, effective contact tracing, mask-wearing, and social distancing are critical for out-break containment and for achieving herd immunity. However, the existing technology solutions, such as contact tracing apps and social-distance sensing, have been met with suspicion due to privacy and accuracy concerns and have not been widely adopted. Without achieving a critical mass of individual users, these personal technologies have been rendered useless. On the other hand, large-scale policy efforts have been complicated, requiring the coordination of federal, state, and local governments and regulation enforcement logistics. However, local communities balance these approaches and are an unrealized, powerful resource to prevent future outbreaks.This paper proposes a novel Crowd Safety Sensing (CroSS) for building a sustainable safe community cluster against COVID-19 and beyond using affordable Internet of Things (IoT) technologies. CroSS monitors social distancing policies to small, focused communities for accommodating efficient technology penetration, greater accuracy, effective practices, and privacy policy assistance. We implemented a social distancing method and integrated it into an edge-based IoT system. The experimental results show that CroSS detects false-positive social distancing cases. © 2021 IEEE.

Mosaic: Modeling Safety Index in Crowd by Detecting Face Masks against COVID-19 and beyond

In addition to rapid vaccination, predicting possible trajectories of the COVID-19 pandemic is critical to health-care-related policy decisions and infrastructure planning. Growing evidence shows that face masks and social distancing can considerably reduce the spread of respiratory viruses like COVID-19. However, the current pandemic trajectory predictions take overly simplified policy input rather than actual observations of face masks and social distancing practices in a crowd. Thus, it is crucial to monitor and understand the extent of masking practices and assess the safety level in a scalable manner. This paper proposes a novel face masking detection system for Modeling Safety Index in Crowd (Mosaic), a Machine Learning (ML)-based approach for detecting masking in a crowd by building new dense mode crowd mask datasets. Mosaic detects, counts, and classifies the crowd's masking condition and calculates spatiotemporal Safety Index (SI) values for each community instead of detecting individual masking cases. SI data can be shared or published to calculate the area-based SI maps (as opt-in data) for assisting effective policy decisions and relief plans against COVID-19. The experimental results show that Mosaic detects various conditions and types of masking states and calculates SI values of a crowd effectively. This paper proposes a novel face masking detection system for Modeling Safety Index in Crowd (Mosaic), a Machine Learning (ML)-based approach for detecting masking in a crowd by building new dense mode crowd mask datasets. Mosaic detects, counts, and classifies the crowd's masking condition and calculates spatiotemporal Safety Index (SI) values for each community instead of detecting individual masking cases. SI data can be shared or published to calculate the area-based SI maps (as opt-in data) for assisting effective policy decisions and relief plans against COVID-19. The experimental results show that Mosaic detects various conditions and types of masking states and calculates SI values of a crowd effectively. © 2021 IEEE.

CATS: Crowd-based Alert and Tracing Services for building a Safe Community Cluster against COVID-19

COVID-19 has been causing several pandemic waves worldwide due to its long incubation period and hostile asymptomatic transmission. Society should continue to practice social distancing and masking in public despite aggressive vaccinations until achieving population immunity. However, the existing technology solutions, such as contact tracing apps and social-distancing devices, have been faced with suspicion due to privacy and accuracy concerns and have not been widely adopted. This paper proposes a novel infection management system named Crowd-based Alert and Tracing Services (CATS) to build a safe community cluster. CATS applies social distancing and masking principles to small, focused communities to provide higher privacy protection, efficient penetration of technology, and greater accuracy. We have designed a smart tag for managing social distancing. We also implemented a Machine Learning (ML)-based face mask tracking system to build non-binary Safety Impact Values (SIV). © 2021 IFIP.

Your Eyes Show What Your Eyes See (Y-EYES): Challenge-Response Anti-Spoofing Method for Mobile Security Using Corneal Specular Reflections

As the need for contactless biometric authentication becomes more significant during COVID-19, and beyond, the popular biometric authentication method for mobile devices, iris detection, and facial recognition confronts various usability, security, and privacy concerns, including mask-wearing and various Presentation Attacks (PA). Specifically, liveness detection against spoofed artifacts is one of the most challenging tasks as many existing methods cannot conclusively assess the user's physical presence in unsupervised environments. Even though several methods have been proposed for tackling PA with motion challenges and 3D mapping, most of them require expensive depth sensors and fail to detect sophisticated 3D reconstruction attacks. We present a software-based face PA Detection (PAD) method named "Your Eyes Show What Your Eyes See (Y-EYES),"which creates challenges and detects meaningful corneal specular reflection responses from human eyes. To detect human liveness, Y-EYES creates multiple screen image patterns as a challenge, then captures the response of corneal specular reflections using the front camera and analyzes the images using lightweight Machine Learning (ML) techniques. Y-EYES system components include challenge pattern generation, reflection image augmentation (e.g., super-resolution), and ML-based analyses. We have implemented Y-EYES as Android, iOS, and web apps. Our extensive experimental results show that Y-EYES achieves liveness detection with high accuracy at around 200 ms against various types of sophisticated PA. Y-EYES liveness detection can be applied for multiple contactless biometric authentications accurately and efficiently without any costly extra sensors. © 2021 ACM.